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ABSTRACT 

The clustering properties of faint Kvega < 24 galaxies are measured in ultradeep J, H and K near-IR images 
of the Hubble Deep Field South (HDF-S), obtained with ISAAC at the VLT. As a function of the ^-magnitude, a 
relatively large clustering amplitude is found up to K = 24, at a level comparable to the measurements at K ~ 19. 
The photometric redshift distribution of K < 24 galaxies extends to z p hot ~ 4-5, and ~ 40% of the galaxies are 
at Zphot > 2. At the highest redshifts, 2 < z p hot < 4, galaxies selected in the rest frame optical (X-band) appear 
significantly more clustered than galaxy populations selected in the rest frame UV (i.e. Lyman Break Galaxies, 
LBGs), in a similar redshift range and with similar number densities. Galaxy clustering depends on the J — K 
color at 2 < z p hot < 4, with the ^-selected galaxies with J—K > 1.7 reaching ro ~ 8 h~ l Mpc comoving. This 
is a factor of 3-4- higher than the correlation length of LBGs with similar number densities, down to < 27, 
and is also larger than the correlation length of ^-selected galaxies with blue J—K < 1.7 colors. Hence at z ~ 3 
a color-density relation is observed which is qualitatively similar to that observed locally. Fluctuations in the 
amplitude of clustering due to cosmic variance may affect our estimates derived from the small HDF-S field, but 
these are unlikely to change our main conclusions. The galaxies with red J—K > 1.7 colors at 2 < z P hot < 4 are 
likely older and more massive galaxies, on average, than LBGs. They were presumably formed in the highest 
density perturbations at early epochs. Semi-analytical hierarchical models do predict the existence of strongly 
clustered populations at z ~ 3, but with at least a factor of 10 lower number density than the one measured. 
The overall properties of this strongly clustered population consistently suggest that they are the progenitors, or 
building blocks, of local massive early-type galaxies and z ~ 1 EROs, close to their major epochs of formation. 

Subject headings: galaxies: evolution — galaxies: formation — cosmology: observations — large-scale 
structure of the universe — infrared: galaxies — galaxies: high-redshift 



1. INTRODUCTION 

Measurements of clustering at large redshifts can be used to 
shed light on the assembly of large scale structure in the Uni- 
verse and to trace the history and evolution of galaxies. 

Locally the Sloan Digital Sky Survey (Stoughton et al. 2002) 
and the "2 degree field Galaxy Redshift Survey" (2dfGRS, Col- 
less et al. 2001) are now providing a detailed description of 
galaxy clustering (Norberg et al. 2002; Zehavi et al. 2002). 
Redshift surveys in the last few years were also able to provide a 
first glimpse of the evolution of galaxy clustering up to z ~ 1 by 
direct real space measurements (LeFevre et al. 1996; Carlberg 
et al. 1997; Hogg et al. 2000), finding a general decrease of 
the clustering strength with redshift. At higher redshift, signifi- 
cant clustering of Lyman Break Galaxies (LBGs) at z ~ 3 (Gi- 

1 Based on observations collected at the European Southern Observatory, 
Paranal, Chile (ESO LP 164.O-0612). 

2 Based on observations with the NASA/ESA Hubble Space Telescope, ob- 
tained at the Space Telescope Science Institute, which is operated by AURA 
Inc, under NASA contract NAS 5-26555. 



avalisco et al. 1998; Adelberger et al. 1998) has been detected, 
implying a large bias for this population of high-z galaxies, in 
qualitative agreement with hierarchical scenarios (e.g. Baugh 
et al. 1998). Ouchi et al. (2001 ; 2003) recently confirmed these 
high bias values on the basis of an extensive survey for LBGs 
at z ~ 4 and for Lya emitting galaxies at z = 4.86, in the Sub- 
am/XMM Deep Survey field. Giavalisco & Dickinson (2001, 
GD01 hereafter) presented evidence of luminosity segregation 
in the clustering of LBGs, with the brighter LBGs being more 
strongly clustered than the fainter ones, possibly consistent with 
the expectations of hierarchical clustering. 

A good approach to address the clustering of high redshift 
galaxies, where spectroscopic redshifts are hard or impossible 
to obtain, is to use photometric redshifts to identify galaxy pop- 
ulations from fields with deep photometric data. As compared 
to the pure LBG selection, this offers the advantage of having, 
in principle, less of a selection bias against reddened or weakly 
star-forming galaxies, and of allowing access to a wider redshift 
range. The most useful fields at this point are the Hubble Deep 
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Fields (HDFs), owing to the very deep and accurate photome- 
try available. Magliocchetti & Maddox (1999) and Arnouts et 
al. (1999; 2002) present clustering measurements of /-band se- 
lected HDF-S galaxies extending up to z p hot ~ 4-5, suggesting 
a trend of increasing clustering for z p hot ~ 2 consistent with a 
positive evolution in the bias. 

Despite an overall agreement between the observations and 
theory, a number of issues remain and deserve attention. Lo- 
cally, it is well established that galaxies of different types clus- 
ter very differently (e.g. Guzzo et al. 1997). At z ~ 1 some ev- 
idence of a significantly different clustering of early-type and 
late-type galaxies is beginning to emerge as well (e.g. Carl- 
berg et al. 1997; Daddi et al. 2002). Since at higher redshift the 
clustering strength of various classes of objects has not yet been 
measured, it is difficult to reconcile the globally measured clus- 
tering at high z with the detailed local measurements of cluster- 
ing. Furthermore, it is still controversial what the present-day 
descendants of LBGs are and so it is difficult to place them in 
the context of a global evolutionary scenario of clustering. Sim- 
ilarly, the decreasing trend of clustering amplitude measured to 
z ~ 1 in redshift surveys is most likely due to the change of 
the galaxy mix with redshift in magnitude selected samples and 
may not reflect physical evolution. 

In this paper, we present the clustering measurement of faint 
/f-selected galaxies in the HDF-S images, obtained for the Faint 
InfraRed Extragalactic Survey (FIRES), and based on very deep 
near-IR observations (more than 33 hours each for the J, H and 
K bands 1 ) carried out with ISAAC at the VET (Labbe et al. 
2003). The FIRES survey (Franx et al. 2000) was designed 
to study the evolutionary properties of galaxies selected in the 
restframe A > 5000 A up to z ~ 3-4, with a selection much 
closer to a selection by stellar mass than by optical/UV light. 
The clustering properties of galaxies in the MS 1054-03 FIRES 
field (Forster-Schreiber et al. in preparation) will be discussed 
in a forthcoming paper. The resulting near-IR selected sam- 
ple of HDF-S galaxies, augmented with the high quality HST 
multiband optical photometry, allows a study of the evolution 
of galaxy clustering that is complementary to the previous ones 
based on optical selection. 

The paper is organized as follow: the data and measurement 
techniques are described in Sect. 2; Sect. 3 describes the angu- 
lar and spatial clustering estimates, for the whole sample and as 
a function of redshift and colors, focusing in particular on the 
clustering properties of I-K and J-K red galaxies. A compar- 
ison with relevant literature data is also presented. We analyze 
the implications of our findings with some modeling in Sect. 4, 
where we discuss the theoretical implications of our detection 
of a strongly clustered population of z ~ 3 galaxies. The sum- 
mary and conclusions of this work are in Sect. 5. We assume 
fl A = 0.7, tt m = 0.3 and H = 100ft km/s/Mpc throughout the 
paper. Magnitudes are given on the Vega scale and distances 
are expressed in comoving units throughout the paper (unless 
otherwise explicitly stated). 

1 The filters used for the observations were the Js and Ks but are referred to as 
J and K in the rest of the paper 



2. DATA AND TECHNIQUES 

2. 1 . The catalog and photometric redshifts 

The reduction, identification of galaxies and the photomet- 
ric measurements, as carried out in the context of the FIRES 
project, are described in detail by Labbe et al. (2003). For 
the present paper a subsample of K-selected galaxies is derived 
from the Labbe et al. (2003) catalog 2 , drawn from the ~ 4.5 
arcmin 2 ISAAC area containing > 95% of the total exposure 
time in the K-band (more than 10 5 s). This selection results in 
a sample whose depth is uniform. The subsample contains 435 
galaxies to the completeness limit of K = 24, determined via 
the K band turnover, with J- and H- band photometry avail- 
able. The seeing of the reduced near-IR mosaics is about 0.45" 
for the J, H and K images. Most of this area (i.e. ~ 4 arcmin 2 ) is 
covered by deep HST images in the f/300, Veo6, hu bands 
(Casertano et al. 2000). The ler sky noise limiting magnitudes 
over 0.7" apertures are 29.5, 30.3, 30.6, 30.0, 28.6, 28.1, 28.1 
for t/300, B450, V606, hi4, J. H, K, respectively (all given on the 
AB magnitude system, Labbe et al. 2003). The very deep and 
high quality imaging available permits an accurate photomet- 
ric redshifts estimate for the sample. Our implementation of 
the photometric redshift measurements is described in detail 
in Rudnick et al. (2001; 2002 in preparation) and Labbe et 
al. (2003). Good agreement is found between the photometric 
and spectroscopic redshifts for the objects with known spec- 
troscopic redshift (Vanzella et al. 2002 and reference therein), 
with an overall Az/(1 +z spe c) ~ 0.09 over the whole range, and 
even better (« 0.05) at z > 2. The spectroscopic redshift is 
adopted when available, i.e. for about 10% of the sample. 

2.2. Angular and real space clustering 

A standard approach, based on the minimum variance Landy 
& Szalay (1993) estimator, is used to measure the two point 
correlation function of galaxies w(8). For a detailed discus- 
sion of the steps involved we refer to Daddi et al. (2000b) 
and references therein. The power-law slopes of the correla- 
tion functions are fixed to the standard value of <5 = 0.8. This 
implies a slope of 7 = 1.8 for the real space two point correla- 
tion function £(r). We generate random samples with typically 
100 times or more objects than in the observed sample. Fol- 
lowing the results of Monte Carlo simulations of Daddi et al. 
(2001), we used as errors for the w(9) measurements in each 
bin a w = [(\ + w)/DD] l l 2 , where DD is the number of observed 
pairs. Note that this error only takes into account effects due 
to the finite number of objects in the sample. However, due to 
cosmic variance, it is possible that there are significant varia- 
tions of w(9) from field to field. To address how representative 
the present measurements are for assessing the average signal, 
we used the analysis of Daddi et al. 2001 (see also Bernstein 
1994, Arnouts etal. 2002): 

a(A)=A 3/2 C 1/2 (1) 

2 available at http://www.strw.leidenuniv.nl/~fires/data/hdfs/ 
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FIG. 1 . — The figure describes the relative fluctuations of the clustering am- 
plitude expected in a 4 arcmin 2 field as due to cosmic variance. The average 
amplitudes of angular clusterinag (< A >) that could have produced the mea- 
sured amplitude (A meas ) as a result of a fluctuation due to cosmic variance can 
be derived from the dotted and dashed lines (1 and 3 sigma levels respectively). 
As can be seen, the curves are highly asymmetric around the measured value, 
generally disfavoring the possibility that the average level of clustering can be 
strongly overestimated as due to the cosmic variance. 



where C is a function of the field geometry (C oc Area -04 ). 
This relation suggest that, when the clustering of a galaxy popu- 
lation is intrinsically large, large fluctuations from cosmic vari- 
ance are expected for the measured amplitudes (Fig. 1), and rel- 
atively small clustering amplitudes can be obtained by chance. 
On the other hand, if the clustering is intrinsically small also 
small fluctuations are expected. Measures of relatively large 
clustering amplitudes are therefore unlikely to arise due to cos- 
mic variance fluctuations from a population with small average 
clustering. 

From a given angular clustering measurement, we use the 
Limber equation to infer the real space clustering (ro) in co- 
moving units and the effective redshift (z, defined as the redshift 
for which ro has been estimated, i.e. ro = ro(z), see Daddi et al. 
2001 for more details). This requires the use of the redshift 
selection function for the examined sample, which we derive 
from the photometric redshift distribution, appropriately con- 
volved with an error distribution function for the photometric 
redshifts. For this we typically used Gaussian functions with 
a z = 0.25 in order to wash out structures in the observed pho- 
tometric distributions. Variations around this value (e.g. from 
a z = 0.1 to 0.5) produce only minimal variations in the esti- 
mated ro, well below the quoted uncertainties. 

A subsample selected for a certain range in photometric red- 
shifts will also contain a certain fraction of galaxies with true 
redshift outside this range. In general these galaxies would di- 
lute the clustering signal. The measured signals can therefore 
be regarded as lower limits to the true signal, in those cases. 

3. RESULTS ON THE ANGULAR AND REAL SPACE 
AMPLITUDE 

In this section we will present the results of our study of the 
clustering of galaxies in the HDF-S as a function of magnitude, 
redshift and color. Table 1 summarizes all the angular clus- 
tering measurements presented in the paper together with the 
relevant derived quantities. 
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FIG. 2. — The sky distribution (right panels) and redshift distributions (left 
panels, smoothed with a Gaussian with a z ~ 0.25) for galaxies to the limiting 
magnitudes K = 22, 23, 24. 



3.1. Clustering of galaxies to K = 24 

Fig. 2 shows the sky and redshift distributions for the galax- 
ies in our survey. The redshift distribution of K < 24 galaxies 
peaks at z p hot ~ 1 and has a tail extending to z p hot ~ 5, becom- 
ing more and more prominent at fainter magnitudes. Its overall 
shape at K < 24 is similar to that derived from the Subaru Deep 
Field (Kashikawa et al. 2003). An inhomogeneous angular dis- 
tribution of galaxies is apparent from these plots. Measure- 
ments of the angular two point correlation functions are shown 
in Fig. 3: the clustering of galaxies is detectable down to the 
faintest levels with S/N ~ 3. The overall amplitude of cluster- 
ing in all cases is low enough (A < 10" 3 , Fig. 3) to allow us 
to neglect the effects of cosmic variance (Fig. 1). These are 
the first measurements of clustering to date extending to mag- 
nitudes K > 22. 

In Fig. 4 our measurements are compared with a collection 
of literature data at brighter magnitudes. The new FIRES mea- 
surements clearly indicate that the amplitude of clustering re- 
mains relatively flat, at the level of the K ~ 19 measurements, 
all the way to K = 24 with only a mild decline. Preliminary indi- 
cations of this behavior were already found at K = 21 .5 by Carl- 
berg et al. (1997, a result also discussed by Roche et al. 1998), 
with a measurement consistent with our own at this bright K 
level. This trend is significantly different from the most recent 
measurements in optical imaging surveys. In the R- and /-bands 
a monotonically decreasing clustering amplitude is observed to 
the faintest magnitudes of R ~ 29 and / ~ 28 (Postman et al. 
1998, Villumsen et al. 1997, McCracken et al. 2001). The 
FIRES galaxies with K < 24 have a median color of I-K ~ 2.4 
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FIG. 3 . — Angular two point correlation function measurements for the galaxies for various K limiting magnitudes. We show the best fit power law w(9) = A t 
(solid line) and the allowed Ict range (dotted lines). 
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FIG. 4. — Plot of the measured angular clustering amplitudes in the FIRES 
survey together with measurements taken from the literature. At bright magni- 
tudes we show PLE models from Roche et al. (1998) with scaling from local 
galaxy clustering. At 20 < K < 24 we show predictions for the clustering of 
/f-selected galaxies based on (dotted line) an evolving model that can repro- 
duce the observed /-band clustering at faint magnitudes and (dashed line) the 
same with enhanced clustering for early-type galaxies with ro = 10 hr l Mpc at 
all redshifts (see Sect. 4 for more details). 



and a median / ~ 25 magnitude 3 . McCracken et al. (2001) mea- 
sure, for galaxies with median / ~ 25, an angular clustering that 

3 Our /-band measurements refer to the /g[4 filter magnitude, expressed in the 
Vega scale. A color term from the standard Cousins /-band may be expected of 
order of 0. 1 magnitudes 



is 3-4 times lower than the one of FIRES galaxies. 

Inverting the angular clustering with the Limber equation, us- 
ing the redshift distributions given in Fig. 2, we find for K < 24 
that the observed level of clustering requires ro = 3.5-4.5 Mpc 
at an effective redshift of z = 1 .3. This value is similar to the typ- 
ical correlation length of the faintest 2dfGRS galaxies at z = 
(Norberg et al. 2002). Because they are intrinsically fainter, the 
galaxies with z p i,ot 1 are expected to have a similar or even 
lower correlation length. The z p i wt > 1 galaxies, for consis- 
tency, should also have a similar or higher correlation length, 
up to Zphoi ^ 3—4 where the tail of the galaxy distribution ex- 
tends, to produce the measured angular correlations. 

These results show clearly that faint ^-selected samples are 
intrinsically more clustered than optically selected samples. The 
interpretation for the clustering trend at very faint K levels will 
be explored further in Sect. 4 with some modeling. 

3.2. Clustering as a function of redshift 

The galaxies with K < 24 were divided in photometric red- 
shift bins in order to measure the redshift evolution of cluster- 
ing. Fig. 5 shows the results. The measurements are noisy, 
because of the low number of galaxies (comparable to previ- 
ous studies of optically selected galaxies, e.g. Magliocchetti 
& Maddox 1999). A comoving clustering with r ~ 3.5^.5, 
that remains constant up to z ~ 3-4, is roughly consistent with 
the measured clustering, in agreement with what is found from 
the inversion of the angular clustering of the whole K < 24 
sample. Our measurements can be compared with the analo- 
gous ones from /-selected samples of faint galaxies in the HDF 
North (Magliocchetti & Maddox 1999, Arnouts et al. 1999) 
and HDF-S (Arnouts et al. 2002), derived using photometric 
redshifts. Even though a detailed comparison as a function of 
redshift cannot be obtained, we notice that the average comov- 
ing correlation lengths of ro ~ 2-3 h~ l Mpc measured for the 
optically-selected galaxies is lower than the average one de- 
rived from our TT-band sample. 
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FIG. 5. — The correlation length of K < 24 galaxy samples as a function of 
redshift. 



A particularly interesting redshift range is the one at z ~ 3, 
where measurements of clustering for LBGs samples are avail- 
able as well. In the HDF-S, Arnouts et al. (2002) measure A = 
9.6±3.0 x 10" 4 and r = 3.2±0.7 IT 1 Mpc for 2.5 < z pho t < 3.5, 
whereas we find A = 36 ± 8.1 x 10" 4 and r = 6.5^g;| h~ l Mpc in 
the same redshift range. Therefore, /f-selected galaxies appear 
to be more clustered at z p hot ~ 3 than /-selected ones, at the 3cr 
level of significance. 

An interesting and more accurate comparison can be per- 
formed with the clustering of LBGs at z = 3, using the most 
up-to-date estimates from GD01. To compare with the LBGs 
we add together all the K < 24 galaxies in 2 < z p hot < 4, and 
find A = 17.1 ±4.6 x 10" 4 and r = 5.5 ± 0.8 h~ ] Mpc. 

GD01 estimate a clustering amplitude of rr> ~ 1 ± 1 h~ l Mpc 
for the fainter HDF sample, with evidence that the amplitude 
depends on galaxy luminosity, and hence galaxy number den- 
sity. Brighter (i.e. rarer) sources have stronger clustering, reach- 
ing r = 5.0±0.7 for the so-called "SPEC sample" with K < 25. 
For a fair comparison to LBG samples we have therefore to 
carefully take into account number-density effects. We plot in 
Fig. 6 the correlation lengths as a function of number-density 
for z ~ 3 LBGs and 2 < z P hot < 4 FIRES galaxies. To derive the 
number density of our samples we divide the observed num- 
ber of galaxies by the volume defined by the area on the sky 
and the FWHM of the photometric redshift distribution 4 . K- 
selected galaxies at 2 < z p hot < 4, have a correlation length of 
ro = 5.5 ±0.8, and are therefore more clustered than LBGs with 
similar number density, which have ro ~ 1 .5-2, based on the 
observed scaling (Fig. 6). To derive the significance of this re- 
sult, we notice that assuming ro ~ 1 .5-2 for the FIRES galaxies 
would result in an angular clustering amplitude A^3x 10~ 4 , 
whereas we observe A = 17.1 ±4.6 x 10~ 4 , therefore the effect 
is at the £ 3a level. In conclusion, there is a strong suggestion 
that -selected galaxies at z ~ 3 have either a larger ro at a fixed 
number density or a higher number density at a fixed ro, with 

4 This corresponds, e.g., to the volume at 2.3 < z < 3.7 for galaxies with 
J-K > 1.7 and 2 < 

Zphot < 4. We note, however, that systematic effects in 
the estimate of the sample redshift distribution would influence both the clus- 
tering and number density estimates, moving the points in Fig. 6 in a direction 
basically parallel to the LBG trend, having thus little effect on the inferred con- 
clusions. 
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FIG. 6. — Comoving correlation lengths for FIRES galaxies at 2 < z p hot < 4 
(filled squares, see labels for all 2 < z phat < 4 galaxies, J-K>1.1 and J-K < 
1.7 galaxies), for LBGs (empty triangles, connected by aline, from GD01) and 
for SCUBA galaxies (star, see text for references) are plotted as a function of 
number density. 



respect to LBGs. 

It is important to stress that this conclusion is robust against 
possible effects due to cosmic variance. In fact, the fluctuations 
due to cosmic variance expected in the HDF-S for a clustering 
amplitude of A £ 3 x 10" 4 , similar to that of LBGs, are about 
10% of such a value (Fig. 1). This cannot produce the much (5 
times) larger clustering amplitude that we have measured. 

3.3. Clustering of red J- K galaxies at 2 < z p hot < 4 

An efficient approach to study the clustering of high redshift 
(i.e. z > 2) galaxies is to select galaxies by their J-K color. 
The effective redshift of J-K color selected samples grows to 
z ~ 3 for the reddest thresholds. The clustering amplitude also 
increases with increasing J-K color threshold (Fig. 7). The re- 
sulting comoving correlation length is also a strong function of 
color, again with values of ro ~ 10 for J-K > 1 .7-2.3, pointing 
to the interesting result that a very strongly clustered population 
of galaxies exists at z ~ 3. 

3.3.1. Color segregation of clustering in 2 < z p hot < 4 

We have measured the angular clustering of galaxies restricted 
to those with photometric redshift estimate 2 < z P hot < 4. Di- 
viding the sample at J- K = 1.7 we obtain two subsamples of 
nearly equal size, containing 49 and 66 galaxies for the red and 
blue subsamples respectively. For the redder subsample with 
J-K > 1 .7 we obtain ro = 8.3 ± 1 .2 (Fig. 8), consistent with the 
trend plotted in Fig. 7. If different redshift ranges are chosen 
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FIG. 7. — The clustering amplitude (top panel), the inferred comoving 
correlation length (center panel, in h~ l Mpc) and the effective redshift are 
given for galaxies redder than the J — K color threshold. The error bar on 
the effective redshift show the standard deviation of the photometric red- 
shift distribution. The power law fit to the angular clustering (top panel) has 
log(A) = 1.00(7 -K)- 4.53 



within 2 < z phot < 4, and/or a J-K threshold redder than 1.7 is 
used, the derived correlation lengths are always in the range 8- 
10 h~ l Mpc, albeit with larger statistical error. The sample with 
J—K < 1 .7 is much less clustered, with r Q = (Fig. 8). The 

difference in angular clustering between the J—K red and blue 
samples is significant at the ~ 3<r level. The strongly clustered 
sample with J-K > 1.7 appears to be the cause of the signifi- 
cantly larger clustering measured at 2 < z p hot < 4 for K-selected 
versus optically selected samples or LBGs. The clustering of 
blue J-K < 1 .7 galaxies at 2 < z p hot < 4 is in fact consistent to 
that of LBGs with similar number densities (Fig. 8). 

The key property for identifying this z ~ 3 strongly clustered 
population appears indeed to be the presence of very red J — K 
colors, irrespective of the properties of the optical continuum 
or of the observed near-IR or optical magnitudes. We note 
that the median K-band magnitudes of the red and blue sam- 
ples are respectively K = 22.7 and K = 22.8, a difference that is 
clearly not significant. The median Veoe magnitude of the J — K 
redder and bluer subsample does appear instead to be signif- 
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FIG. 8. — The sky distribution of 2 < Zphot < 4 galaxies (top-right panel). 
Objects with J -K < 1.7 are shown as crosses, while filled squares are for the 
ones with J-K > 1.7. The remaining panels show the measured two-point 
correlation functions for the different subsamples at 2 < Zphot < 4. 



icantly different, with Vso6 = 26.4 and Veo6 = 25.6 for the red 
and blue samples respectively. This difference in V(,Q6, which 
is a consequence of the ^-selection of the sample, is not the 
cause of the measured clustering segregation, as it would bias 
the bluer-brighter (and not the redder-fainter) subsample toward 
larger clustering (GD01). Finally, we tested that no measur- 
able color dependence of clustering can be found if galaxies at 
2 < z p hoi < 4 are split by their U— V, V— J or even V -K color. 

The large clustering inferred for the J -K reddest galaxies at 
2 < Zphot < 4, together with the small area of the HDF-S field, 
consistently imply a highly non uniform redshift distribution 
(see e.g. Broadhurst et al. 1992; Cohen et al 1996; Daddi et 
al. 2001; 2002). To investigate this aspect, Monte Carlo simu- 
lations were used to produce random realizations of our survey. 
A population of galaxies with rrj = 8 h~ l Mpc was extracted with 
a flat selection function between 2 < z < 4 over a 4 arcmin 2 
area, following the recipes of Daddi et al. 2001. The resulting 
redshift distributions (Fig. 9) are very spikey. Fig. 9 also shows 
the observed photometric redshift distribution of galaxies with 
J-K > 1.7 and 2 < z p hot < 4, which suggests the presence of 
three main galaxy concentrations at z p hot ~ 2.4, 3 and 3.5, in 
qualitative agreement with our Monte Carlo simulations. While 
systematic errors in the photometric redshift determination may 
still result in an incorrect determination of the real redshift of 
the spikes, there is evidence that these features reflect real red- 
shift segregations. In fact, we tested that restricting the clus- 
tering measurements to redshift ranges covering such features 
imply an enhanced angular clustering. This effect is expected 
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FIG. 9. — The photometric redshift distribution (in 0.1 redshift bins) of 
J-K > 1.7 galaxies at 2 < z p hot < 4 is compared to random (Monte Carlo) 
realizations of a population at 2 < z < 4 with ro = 8 /T 1 Mpc (small boxes, Az = 

0. 01 binning). When account is made for photometric redshift uncertainties, 
the observed distribution appear consistent with the Monte Carlo realizations, 

1. e. it appear to contain redshift spikes. 



only if the galaxies in the photometric redshift spikes are spa- 
tially associated. 

As a further confirmation, van Dokkum et al. (2003) present 
in a companion paper spectroscopic confirmation of 5 galax- 
ies with J-K > 2.3 at 2 < z < 4 (selected from the MS1054- 
03 field), with 3 of them belonging to a single redshift spike, 
in qualitative agreement with the estimates of large clustering 
presented here. We also note that Saracco et al. (2001) using 
IR data on HDF-S and "Chandra Deep Field South" found that 
J-K > 2.3 galaxies are unevenly distributed, with their sur- 
face density varying by over a factor of two over the two fields, 
qualitatively consistent with the results presented here. 

In Fig. 8 (upper-left panel) an excess of very close pairs (with 
separations ~ 2 arcsec) of J-K red galaxies is apparent in the 
measured two point correlation function. A similar excess at 
short separations was noted by Ouchi et al. (2001) in their sam- 
ple of z ~ 4 LBGs. This feature is potentially interesting: 2 
arcsec at z ~ 3 corresponds to ~ 50 kpc and suggest a merg- 
ing destiny for these galaxies (if they are indeed at identical 
redshifts). We find that the excess mainly arises because of 2 
triplets of very close separation galaxies in our sample, yield- 
ing a large number of close pairs: 5 of these have photomet- 
ric redshift estimates placing them in the z p i wt = 3.5 spike of 
galaxies and J-K > 2.1 colors (i.e. at the extreme range). 
The feature disappears if the clustering is measured exclud- 
ing these "triplets" galaxies. At the same time, such an excess 
at a very small scale may suggest a correlation function slope 



steeper than the one assumed (7 > 1.8), which would be con- 
sistent with measurements for local early-type galaxies (Guzzo 
et al. 1997). Our measurements cannot significantly constrain 
the slope of the correlation functions, because of the relatively 
small range of probed angular scales. Previous measurements 
for LBGs suggest possible values of 7 = 1.5-2.1 (Giavalisco et 
al. 1998, GD01, Porciani & Giavalisco 2002) that would all 
be basically consistent with our data. We notice, however, that 
for the case of J-K > 1.7 galaxies in 2 < z p hot < 4 (with es- 
timated ro = 8.3 h~ l Mpc for 7 = 1.8), changing 7 to 1.5(2.1) 
increase(decrease) the inferred ro by only 7.5%(8.5%). 

3.3.2. J—K red galaxies at 2 < Zphot < 4 and LBGs 

It is relevant at this point to compare in detail the number 
density and clustering of this strongly clustered z^3,J-K> 
1.7 population to the LBGs (GD01, see also Adelberger et al. 
1998, Giavalisco et al. 1998, Porciani & Giavalisco 2002). 
LBGs with a comoving density similar to this population are 
expected to have ro ~ 2-2.5 (Fig. 6), that would produce an an- 
gular clustering of A < 5 x 10" 4 , assuming the observed photo- 
metric redshift distribution of our red z ~ 3 population. In con- 
trast, for the red ^-selected galaxies at z ~ 3, A = 38 ± 10 x 10" 4 
is measured. Hence the red galaxies are more clustered than 
LBGs with the same number densities at the 3.3<r level. The 
results appear again robust with respect to cosmic variance in 
the clustering. To quantitatively address the point we analyze 
in Fig. 10 the constraint on the average correlation length of K- 
selected galaxies with J-K > 1 .7 colors, allowing for both the 
measurement error and the cosmic variance fluctuations (added 
in quadrature). 

The selection of FIRES galaxies is based solely on the K- 
band apparent magnitude. The standard z ~ 3 LBG selection is 
based instead on the presence of an f/-band dropout and blue 
optical colors, which in principle is biased against galaxies with 
low star-formation rates or those that are highly obscured by 
dust. It is important, therefore, to compare the properties of the 
samples selected by the two criteria. 

We adopt the HDF-LBG definition by GD01, 

(1/300-^450) > l-0 + (#450-V606), 
(t/300-#450) > L6, (B45O-V6O6) < L2 

(AB magnitudes). Restricting ourselves to the redshift range 
where the LBG selection is effective, i.e. 2.0 < z p hot < 3-3, we 
find that about 30% (10/35) of the J-K > 1.7 galaxies satisfy 
these criteria. The remaining galaxies are still consistent with 
being LBGs, all having quite blue < (#450- V6O6X4B < 1 colors 
(within 2a), but are very faint in the optical and have lower 
limits to the U30Q-B45Q colors that do not allow them to be 
individually classified as LBGs even in the very deep HDF-S 
data. However, we find that, by stacking together those that 
do not individually qualify as a LBG, we obtain average colors 
fully consistent with those of a U -dropout galaxy, i.e. an LBG. 

Even if the J — K red galaxies at 2 < z P hot < 4 and the LBGs 
may not be distinguishable from the rest-frame UV colors, the 
two criteria define populations with quite different properties 
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FIG. 10. — Significance levels for the clustering of faint X'-selected galaxies 
with J—K > 1.7, accounting for both the measurement error and the cosmic 
variance fluctuations. The correlation length is greaten than 5.3(3.6) h~ l Mpc 
at the 2(3) sigma level. A limit on the correlation length of LBGs with similar 
number density is plotted for comparison. 



for at least two main reasons: significantly different clustering 
properties and significantly different rest-frame optical colors 
(i.e. the observed J- K). The overlap between our sample of 
z ~ 3 red galaxies and the LBG samples selected in the HDF-S 
by GD01 to Veoe < 27 is in fact small. 

The color segregation of clustering between J-K red galax- 
ies on one side, and bluer /f-selected galaxies or LBGs on the 
other side, appears as a much stronger effect than the luminosity 
segregation among LBGs discussed by GD01. This is similar 
to what observed for local galaxies, where color (i.e. morpho- 
logical) segregation of clustering is strong and well established 
at least since Davis & Geller (1976), while a convincing mea- 
sure of the mild luminosity dependence of clustering at z = 
has been achieved only in recent years (Norberg et al. 2002). 

3.3.3. J-K red galaxies at 2 < z p hot < 4 and SCUBA sources 

Sub-mm bright galaxies detected by SCUBA are another in- 
teresting classes of sources of which a significant fraction is 
expected to be at z > 2 (e.g. Smail et al. 2002a and reference 
therein). Scott et al. (2002) and Webb et al. (2002) recently 
report a « 2er detection of angular clustering, and infer a cor- 
relation length of r = 12.8 ±4.5 h~ l Mpc. Such high level 
of clustering is consistent with what we find for the red J-K 
galaxies at 2 < z p hot < 4. 

However, the number density of SCUBA sources is estimated 
in the range lO^-lO -5 h 3 Mpc" 3 (Scott et al. 2002; Smail et 
al 2002b), about 2 order of magnitudes lower than that of the 
red J-K galaxies at 2 < z p hot < 4 (Fig. 6). Given the large 
difference in these number densities, it is possible that these 
SCUBA sources are a subset of the red J- K galaxies. 

Studies of SCUBA galaxies so far have mainly concentrated 
on their R-K and/or I-K colors (i.e. checking if they qualify 
as EROs), often yielding ERO-like colors. It would be very 
interesting to determine the J-K colors of a statistical sample 
of SCUBA sources to test whether the SCUBA galaxies are in- 
deed a subset of the new class of J-K red objects highlighted 
by our deep TT-band FIRES survey. 




2 2.5 3 3.5 
(I-K) min 

FIG. 1 1 . — The same of Fig. 7 but for the I-K color threshold. 



3.4. Clustering of faint EROs at 0.8 < z p hot < 2 

For completeness, we present in this paragraph the clustering 
of ^-selected galaxies as a function of the I-K color. Galax- 
ies with red I-K colors are classified as extremely red objects 
(EROs), and allow one to address redshift ranges at 1 £ z ~ 2 
(Cimatti et al. 2002a), in agreement with our photometric red- 
shift estimates. Daddi et al. (2000b) found a monotonic power- 
law increase of log(A) as a function of R-K color for K < 18.8 
galaxies. For the FIRES galaxies that are on average 5 magni- 
tudes fainter we still detect a general correlation between clus- 
tering and I-K color. (Fig. 11). A difference is that for the 
FIRES galaxies the clustering amplitude shows a plateau for 
2.5 < I-K < 3.5 and increases again toward redder I-K colors. 
Using the photometric redshifts we find that the angular ampli- 
tude implies a correlation length consistent with ro ~ 4-5 h~ l 
Mpc for the galaxies with I-K selection threshold lower than 
3.5, applied at effective redshifts 1 < z < 2 (see also Fig. 1 1). 

Galaxies with I-K > 3.5 show a stronger amplitude, and are 
consistent with r ^ 8 h~ x Mpc. The I-K > 3.5 selection cri- 
terion is close to the ERO selection criterion R-K > 5. Even 
though our measurements extend in principle up to 5 magni- 
tudes deeper, very similar values are found for the clustering of 
Z ~ 1 EROs (Daddi et al. 2001, 2002; Firth et al. 2002, Roche 
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FIG. 12. — The comoving correlation length inferred for EROs (z sa 1), as 
a function of the sample limiting magnitude. The fitted line is based on the 
scaling b/b* = 0.85+0.15 *L/L* (Norberg et al. 2002). 



et al. 2002, Miyazaki et al. 2002). However, we note that the 
clustering signals here predominantly arise from red galaxies 
with 1 < z p hoi < 2 that have typically K < 22 and are therefore 
only ~ 2 magnitudes fainter than the EROs analyzed by Roche 
et al. (2002) and Miyazaki et al. (2002). 

The K20 survey (Cimatti et al. 2002a, Daddi et al. 2002) 
established from deep VLT spectroscopy that K ~ 19-20 EROs 
come in two different flavors: early-type galaxies and reddened 
starbursts. About 12(3) of the FIRES galaxies with I-K > 3.5 
have K < 20(19) and would have been inserted in the K20 sam- 
ple, i.e. only 15%(4%). Recent studies of the faints > 20 ERO 
population suggest a non-negligible fraction of dusty starbursts 
(Wehner et al. 2002, Smail et al. 2002b). Our faint I-K > 3.5 
EROs at Zphot < 2 typically have 1 .6 < J-K < 2.0, which may 
favor these to be predominantly early-type galaxies (Pozzetti & 
Mannucci 2000). No galaxies with I-K > 3.5 and z p hot < 2 
are found at K > 22. This is probably not an effect of cos- 
mic variance, as Yamada et al. (2001) notice a similar behavior 
at K > 22 in the HDF-N and 53W002 field. A low density 
of K > 22 EROs would be expected assuming a "bell shaped" 
early-type galaxy luminosity function, a feature observed up to 
z = 1 (Pozzetti et al. 2002, cfr. also Totani et al. 2001 and 
Smail et al. 2002b on this point). Again, this behavior is ex- 
pected for faint early galaxies, but seems unlikely for starbursts. 
In addition, Daddi et al. (2002) present evidence that the ERO 
clustering at z < 2 is mainly due to the early-type galaxies. The 
dusty EROs are found to be very weakly clustered, but their 
diluting effect on the overall clustering is weakened by a non- 
negligible cross-correlation between the two galaxy types (see 



however Miyazaki et al. 2002 for a somewhat different result). 

Even if the uncertainties here are much larger than for z > 2 
red galaxies discussed in the previous sections (mainly because 
the clustering amplitudes for EROs are measured at the « 2.5c 
level of significance only) we find evidence, in conclusion, for 
faint L < 0.1L* 5 early-type galaxy clustering that is still con- 
sistent with a large ro of 10 h~ l Mpc at 1 £ z ^ 2. This high 
level of clustering of faint EROs suggests a weak luminosity 
dependence of early-type galaxy clustering at z ~ 1 . In the lo- 
cal universe it has been recently found that the bias of galaxies 
(and in particular also of early-type galaxies) follows the rela- 
tion b/b* = 0.85 + 0.15 x (L/L*) (Norberg et al. 2002). We plot 
in Fig. 12 the EROs clustering measurements taken from the lit- 
erature together with the best-fitting Norberg-like law. For the 
FIRES EROs we report the measurements obtained for galaxies 
with I-K > 3.5 and 0.8 < z pho t < 2, i.e. r = 9.7 ± 2 /T 1 Mpc. 
Only the Daddi et al. (2001) data explicitly take into account 
cosmic variance of clustering, while it is actually the highest 
precision measurement (S/N ~ 8, vs S/N <~ 2-3 for most of the 
other measurements). The observed trend is in agreement with 
the local scaling behavior of the clustering with luminosity. Our 
new data at faint luminosity is consistent with the large overall 
normalization of the relation, with ro(L = L* ,z ~ 1) ~ 11 h~ l 
Mpc, larger than local values. This again favors the hierarchi- 
cal clustering scenario for the evolution of early-type galaxies 
clustering (see Daddi et al. 2001 for a more detailed discussion 
of this point). In principle, because of clustering evolution, the 
local relation for the luminosity dependence of the bias may be 
expected to change with redshift. In particular, at 1 < z < 2, 
a steeper dependence on luminosity should be expected at the 
bright end (L > L*), both for "galaxy conservation" and "galaxy 
merging" scenarios. This cannot be ruled out, of course, on the 
basis of the present data: the K = 19 measurements were in 
fact shown to be lower limits, because of the diluting effect of 
the dusty EROs on clustering (Daddi et al. 2002). Anyway, it 
seems probable that a quite large ro ^ 15 h~ l Mpc may be ex- 
pected for very bright K ~ 17-18 early-type galaxies at z ~ 1. 
It is natural to identify these brightest EROs with the short lived 
powerful radio galaxies that have ro consistent with high values 
(Overzier et al. 2002). 

4. DISCUSSION AND MODELING 

4.1. An empirical model for the clustering at faint K 
magnitudes 

We will first discuss the evolution of the clustering of galax- 
ies selected in the /f-band. With a number of simple assump- 
tions we will first model the relatively flat dependence of the 
angular clustering amplitude on the K magnitude down to the 
faint K = 24 magnitude (see Fig. 4). 

Galaxies with ^-magnitudes in the range 15 < K < 18, i.e. 
down to the limit where the slope of the amplitude versus K- 
band magnitude changes, have a median redshift that scales 
with ^-magnitudes and is in the range of 0. 1 to 0.4 (Songaila et 
al 1994, Cowie et al. 1996). The number counts in this range 

5 assuming K(L* , z = 1) ~ 19 from Daddi et al. 2000a 
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can be directly predicted on the basis of the local luminosity 
function (Kochanek et al. 2001). In this regime the dependence 
of the clustering amplitude on limiting magnitude can be de- 
scribed using a simple scaling relation as a function of depth 
reached (Peebles 1980). This is due to the fact that, within this 
limited redshift range, clustering evolution is very mild (Baugh 
et al. 1996). At fainter magnitudes the tail of high redshift 
galaxies becomes significant (Fig. 2) allowing studies of the 
redshift evolution of clustering. 

Assuming a comoving clustering of ro ~ 4 h~ l Mpc at all red- 
shifts (consistent with Fig. 5) and using the observed N(z), we 
can reproduce correctly the inferred trend of amplitude versus 
K magnitude of Fig. 4. However, faint K-band selected sam- 
ples are a mix of at least two different populations with differ- 
ent clustering evolution properties. In addition to the strongly 
clustered red galaxies it contains a significant fraction of poorly 
clustered blue galaxies in common with the optically-selected 
ones. McCracken et al. (2001) find that the angular clustering 
amplitude as a function of the /-band magnitude (22 < I AB < 25) 
can be reproduced by a strongly evolving population with ro(z = 
0) = 4.3 h~ l Mpc and e = 1 . Here they used the standard "empir- 
ical" parameterization (e.g. Peebles 1980) for the evolution of 
the 2-point correlation function £(r, z) = £(r, 0)( 1 +z) 7_3+e , where 
the free parameter e determines the clustering evolution. In this 
scenario the comoving correlation length at z ~ 1 has already 
dropped by 60% to a value of ro ~ 1.8 h~ l Mpc, consistent 
with measurements at this redshift by the CFRS (LeFevre et 
al. 1996), and decreases even further at higher z. Adopting the 
McCracken et al. evolution for the clustering and using the ob- 
served photometric redshift distribution in 20 < K < 24, Fig. 4 
(dotted line) predicts clustering at faint K magnitudes which is 
nearly an order of magnitude lower than observed in the FIRES 
survey. The McCracken et al. parameterization of clustering for 
faint /-band selected galaxies would also be inconsistent with 
the evolutionary trend inferred for ro as a function of z (Fig. 5). 

/if -band selected samples at high-z must therefore contain a 
more clustered population, to boost the Zf-selected correlations, 
in addition to the galaxies in common with optically selected 
samples. It is natural to identify massive early-type galaxies 
with this population. Due to an intrinsically old and passive 
stellar population leading to large K-corrections, these galaxies 
have red optical to near-IR colors at least to z ~ 2. Early type 
galaxies should thus be more prominent in near-IR selected 
samples, while they could escape optically selected samples of 
comparable depths. 

Moreover, the clustering of red early-type galaxies at z ~ 1 
has been recently measured (Daddi et al. 2000b, 2001, 2002, 
Firth et al. 2002, Roche et al. 2002) to be very strong with 
ro ^ 10 h~ l Mpc, a value comparable to or even greater than 
that in the local universe, consistently with this paper's findings. 

In the light of these recent developments, we calculate pre- 
dictions for the Zf-magnitude dependence of clustering based on 
a two component model containing: (1) a population of early- 
type galaxies with a redshift distribution derived assuming a 
negligible number density evolution from z = 0, following the 
Daddi et al. (2000a) model and with a constant comoving clus- 



tering of ro = 10 h Mpc, involving typically 15% of the galax- 
ies at K = 20-24; and (2) the remaining galaxies with the rapidly 
decreasing clustering derived by McCracken et al. (2001), hav- 
ing ro(z = 0) = 4.3 ft -1 Mpc and e = 1. The cross correlation be- 
tween the two samples is assumed to have a correlation length 
equal to the geometrical mean of the ones of the two distinct 
populations (as expected in the case of two differently biased 
realizations of the same underlying matter distribution). We 
show in Fig. 4 that this two component model reproduces the 
correlations very well, suggesting that a galaxy population with 
large (ro ~ 10 h~ l Mpc) clustering extending to high redshift 
(z ~ 3) can explain the observed flattening. We recall that our 
measurements of color selected subsamples of K < 24 galaxies 
confirm the presence of such strongly clustered populations at 
1 ... z .,. 4. 

4.2. On the nature of the strongly clustered population at 
z~3 withJ-K> 1.7 

The main result of this study is the detection of a population 
with ro ~ 8 h~ l Mpc at z ~ 3. We recall that there is much 
supporting evidence for the existence of such a highly clustered 
population: 

• The large clustering amplitude measured for all galaxies to 
K = 24 requires the presence of galaxies with strong clustering 
at z > 2. 

• The clustering of 2 < z p hot < 4 /^-selected galaxies is high, 
and larger than that of optically selected HDF galaxies and of 
LBGs over similar redshift ranges and with similar number den- 
sities. 

• We detect a well defined trend of increasing angular clustering 
with an increasing minimum J—K color threshold for the whole 
K < 24 sample, measurable up to the threshold of J — K > 2.3. 
With the use of photometric redshifts we find that such a trend 
implies a growth of the spatial clustering with ro £ 8 h~ l Mpc 
for J—K > 1 .7, again holding at an effective redshift z <~ 3 

• Preselecting galaxies with photometric redshifts 2 < z p hot < 4, 
we detect significant color dependence of clustering at those 
redshifts. If the sample is split at J-K = 1.7, r = 8.3 ± 1 .2 h~ l 
Mpc for the J-K > 1 .7 sample at 2 < z p hot < 4 

• We find apparently significant spiky structure at 2 < z < 4 in 
the photometric redshift distribution, as required by the large 
inferred correlation length. 

• There is evidence of strong field to field variation of faint J—K 
red galaxies, between HDF-S, HDF-N and CDFS (Labbe et al. 
2003, Saracco et al. 2001, this work) 

• There is preliminary confirmation of strong redshift space 
clustering of red J-K galaxies at z > 2 (van Dokkum et al. 
2003) 

The question that now arises is which is the main origin of 
the observed J-K > 1 .7 colors of this strongly clustered popu- 
lation. The redder J-K galaxies may be older and/or more mas- 
sive, may be heavily reddened by dust or may have prominent 
lines (e.g. H a ) falling in the K band. Indeed, van Dokkum et 
al. (2003) detect emission lines in 4/5 of the bright J—K > 2.3 
spectroscopically confirmed objects at z > 2, and in particular 
AGN lines in 2/5. Their contribution to the broad band col- 
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ors is however estimated to be small, £ 10%. Furthermore, 
the objects with red J — K > 1.7 colors at 2 < z P hot < 4 have 
generally a relatively flat and blue spectral energy distribution 
blueward of the /-band (with median (B-I)ab = 1 .01 fully con- 
sistent with the (B-I) colors of ordinary LBGs), suggesting the 
presence of a break between the J and K bands and disfavoring 
strong dust reddening in most cases. We notice that at 2 < z < 4 
the J- and K- bands sample around the rest frame 3000 A and 
5500 A regions respectively, thus targeting a region around the 
4000 A break. This suggest that the most likely interpretation 
of the peculiarity of this J-K red population is an age effect. 
This agrees with a detailed modeling of the SEDs of the red- 
dest (J-K > 2.3) galaxies with z p hot > 2, that is presented in a 
companion paper (Franx et al. 2003). 

Recently, some modeling of the properties of galaxies at z « 
3 have also been presented for LBG samples having both near- 
IR imaging and spectral observations available (Shapley et al. 
2001, Papovich et al. 2001). Shapley et al. describe both J- 
and /T-band observations, thus allowing for comparison with 
our work, and estimate the best fitting properties of a sample 
of spectroscopically confirmed LBGs at 2 < z < 3.5, includ- 
ing the dust reddening, star-formation rate, stellar mass and 
age (intended as the time elapsed since the onset of a conti- 
nous star-formation). We extract a subsample of 40 LBGs from 
the work of Shapley et al. (2001) having all the necessary in- 
formation available, including the J-K color and spectroscopic 
redshift. We find upon dividing the Shapley et al. galaxies at 
the J-K = 1.7 color, that the reddest galaxies turn out to have 
a larger median mass (3 x 10 10 M Q vs 10 10 M Q ), larger median 
age (572 Myr vs 286 Myr) but similar median star-formation 
rate (<~ 45M Q /yr) and extinction properties (E(B-V)^ 0.17). 
This is in agreement with our qualitative considerations pre- 
sented above. A Kolmogorov-Smirnov test shows that the dif- 
ference in the mass distributions in the two samples is signifi- 
cant at the > 99% confidence level, while for the age the signif- 
icance is at the 85% level. It should also be noted, however, that 
the typical error on the J-K color is quite large ( £ 0.4 magni- 
tudes), resulting in some mix in the samples, so that the intrinsic 
trends and differences could be even stronger than found. 

While doing this comparison, we have to keep in mind that 
these Shapley et al. LBGs are brighter than ours (with a median 
K <~ 21 .4 versus K ~ 22.7 for our 2 < z p hot < 4 galaxies), and 
that the Shapley et al. sample was optically selected, and sub- 
sequently observed in the near-IR. The differences in the mass 
and age for J-K red and blue subsamples of LBGs are in fact 
a consequence of strong correlations of the fitted mass with the 
apparent K magnitude (i.e., more massive LBGs are brighter in 
K) and of the fitted star-formation rate with the optical mag- 
nitudes (LBGs with larger star-formation rates are brighter in, 
e.g., R). Rather interestingly, the Shapley et al. (2001) model- 
ing ultimately suggest that the ^-selected J-K red galaxies are 
more massive and relatively older than LBG samples selected at 
comparable depth. This imply a low star-formation rate per unit 
mass for this population, that appears therefore to be relatively 
evolved and, thus, presumably formed at the highest density 
peaks in the matter distribution at significantly earlier epochs. 



Retrospectively, if we have a way to isolate the oldest and most 
massive galaxies in a sample (at any redshift), then it is natural 
to expect a larger clustering for this population, with respect to 
the other younger and less massive galaxies. This gives a rather 
consistent way of interpreting the significantly larger clustering 
of J-K > 1.7 galaxies at 2 < z p hot < 4, with respect to LBGs 
and bluer J-K < 1 .7 objects. 

Our findings suggest therefore that a color-density relation, 
similar to that observed in the local universe (i.e. driven by the 
star-formation rate per unit mass, Dressier 1980), was in place 
since early epochs (at least at z ~ 3) and is therefore not simply 
a product of gravitational growth of clustering at lower redshift. 

4.3. Theoretical models and the existence of a strongly 
clustered population at z ~ 3 

For the clustering evolution of early-type galaxies the semi- 
analytical hierarchical model by Kauffmann et al. (1999) pre- 
dict a large and nearly constant comoving clustering up to z ~ 3, 
matching our observational results well, if indeed this strongly 
clustered population at z ~ 3 is evolving into early-type galax- 
ies. The same hierarchical models, however, require a rapid de- 
cline in their number density to z ~ 3, at least for the most mas- 
sive systems. We have explored this point by analyzing the pop- 
ulation of galaxies included in the GIF simulations 6 based on 
the Kauffmann et al. (1999) models. The corresponding simu- 
lated catalog at z = 2.97 is limited to objects with M > 10 10 M Q , 
and we find that populations with large ro £ 8 h~ l Mpc can be 
selected with different criteria (e.g. the most massive, or the 
most star-forming), but these populations are typically ^ 10- 
100 times less abundant in number density than the population 
we have identified. Scaling from the mass estimates of Shapley 
et al. (2001), and consistent with our own estimates (Rudnick 
et al. in preparation), it seems likely that roughly 30-50% of 
our 2 < Zphot < 4, J-K > 1.7 galaxies have M > 10 10 M Q , sug- 
gesting that a discrepancy with the Kauffmann et al. (1999) 
modeling exists. 

In the hierarchical framework a joint analysis of the clus- 
tering and number density of a population of galaxies can be 
used to constrain their halo occupation function (e.g. Wechsler 
et al. 2001). From Mo & White (2002), a comoving corre- 
lation length of ro > 8 h~ l Mpc at z = 3 is expected for dark 
matter halos with M > 1O 13 M in the ACDM models. Such 
halos have a comoving density <~ 10~ 4 h 3 Mpc" 3 , a factor of 
several tens lower than the number density we estimate for the 
red J-K > 1.7 galaxies at z ~ 3 (Fig. 6). Even allowing for 
a somewhat lower clustering reflecting the observational un- 
certainties we must conclude that if our estimates are correct 
they can be reconciled with the hierarchical clustering scenario 
only if large occupation numbers are characteristic of this pop- 
ulation. This is not unexpected in numerical simulations: for 
example White et al. (2001) estimate the existence of about 
10 sub-halos for each DM halo already at M ~ 1O 12 M , with 
the number of sub-halos increasing almost linearly with halo 
mass. However, it is expected from theory that many of these 
sub-halos would not produce a visible galaxy (e.g. Primack et 

6 http://www.mpa-garching.mpg.de/GIF/ 
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al. 2002). 

On the other hand, the existence of numerous sub-halos could 
perturb the two-point correlation function at small scales. In 
Sect. 3.3.1 we actually do find hints for a small-scale excess 
in the correlation functions that could be due to such an effect. 
However, a detailed modeling of these aspects within the "halo 
model" in the ACDM scenario is beyond the scope of this paper. 

4.4. Detecting the progenitors of early-type galaxies ? 

In a recent paper, Moustakas & Somerville (2002) present a 
detailed study of the clustering and number density properties 
of local giant ellipticals, EROs and optically selected LBGs try- 
ing to establish a possible evolutionary link between these pop- 
ulations, in the framework of hierarchical CDM models. Inter- 
estingly they estimate that both local giant ellipticals and EROs 
are hosted typically by M <~ 10 13 M Q halos and are character- 
ized by non-unity occupation numbers, while they constrain the 
characteristic halo mass of (standard, optically selected) LBGs 
to be M ~ 1 1 1 Mq or lower, disfavoring the possibility that they 
are the direct progenitors of the former classes. This conclusion 
was also reached by Daddi et al. (2001) on an empirical basis. 

Moustakas & Somerville (2002) also provide some predic- 
tions for the properties of the progenitors of present-day ellip- 
ticals and z ~ 1 EROs. These progenitors are expected at z ~ 3 
to be hosted by M ^ 10 13 M Q halos (similar to their lower z de- 
scendants) and therefore to show large clustering with ro in the 
range 7-15 h~ l Mpc, and to be characterized by large halo oc- 
cupation numbers. These properties are in remarkable agree- 
ment for what is inferred for /^-selected J-K > 1 .7 galaxies at 
2 < Zphoi < 4. 

The large R-K colors and spectral properties of z ~ 1 early- 
type galaxies (Cimatti et al. 2002a), as well as the fundamental 
plane studies of ellipticals up to z < 0.8 (van Dokkum et al. 
1998), suggest that a substantial fraction of the stars ending up 
in local ellipticals and EROs were already in place at z ^ 3. 
This is consistent with, and requires, the existence of relatively 
old and massive galaxies at z > 2, that we may have detected. 

It is suggestive to interpret both the empirical and the theoret- 
ical suggestions concluding that we may have located the pro- 
genitors of local massive early-type galaxies and of z ~ 1 EROs 
in the subsample of very red, Zf-selected, J-K > 1.7 galaxies 
at z ~ 3. As the inferred halo occupation number as well as the 
number density of our faint red z > 2 galaxies are larger than 
that of massive early-type galaxies at z = or of z ~ 1 bright 
EROs, merging could be required to reduce both, and the to- 
tal mass of each finally formed galaxy would correspondingly 
grow. 

5. SUMMARY AND CONCLUSIONS 

The clustering properties of K < 24 galaxies in the HDF-S 
field have been analyzed. The main results can be summarized 
as follow: 

• We have produced a first assessment of the clustering of galax- 
ies as a function of K magnitude up to K = 24. Whereas the 
clustering amplitude flattens, as already known, down to mag- 
nitude K <~ 19, it then remains surprisingly high with only a 



slight further decline out to K = 24. 

• Modeling of the clustering of ^-selected galaxies at 20 < 
K < 24 requires a strongly clustered sub-population and is con- 
sistent with a picture in which early-type galaxies are strongly 
clustered, with ro ~ 10 h~ x Mpc h~ l Mpc, all the way to z ~ 3. 

• We have analyzed the clustering of galaxies in photometric 
redshift bins, and detected strong clustering at 2 < z p hot < 4. 
The clustering of the K band selected sample is stronger than 
that of LBG's of similar number density. 

• At redshifts higher than 2, the clustering amplitude depends 
on the J-K colors of the galaxies. Redder galaxies have stronger 
clustering. Galaxies with J-K > 1.7 and 2 < z p hoi < 4 have 
ro ~ 8 h~ x Mpc. These galaxies likely have high ages and high 
mass-to-light ratios (see also Franx et al. 2003). 

• The color dependence of the clustering suggests that a color- 
density relation, qualitatively similar to that observed locally, 
was already in place at those early epochs. 

• A redshift distribution with prominent spikes is predicted for 
the /f-selected population of HDF-S galaxies with 2 < z p hoi < 4, 
particularly for the redder galaxies with J-K > 1.7, in agree- 
ment with our photometric redshift analysis. 

• In a CDM framework, these J-K red galaxies at 2 < z < 4 
would be hosted by M ^ 10 13 M Q halos, with large occupation 
numbers (i.e. within sub-halos). Semi-analytical models ap- 
pear to severely underestimate the number density of strongly 
clustered, ro ^ 8 h~ l Mpc, galaxies at z ~ 3, that are as numer- 
ous as faint LBGs. 

• We have discussed the properties of this newly discovered 
population of strongly clustered z ~ 3 galaxies, including num- 
ber densities and clustering, with the plausible conclusion that 
a direct evolutionary trend exists between these J-K red z ~ 3 
galaxies on one side and EROs at z ~ 1.5 and local massive 
early-type galaxies on the other side. 

Over the last several years the popular scenario in which 
bulge-dominated galaxies form at relatively low redshift by the 
merging of full-sized spirals has encountered increasing diffi- 
culties in accounting for the growing body of observational ev- 
idence (see e.g., Renzini & Cimatti 1999; Peebles 2002). In 
particular, in semi-analytical simulations this scenario fails to 
produce enough red galaxies at z ^ 1 (Daddi et al. 2000a; Smith 
et al. 2002; Cimatti et al. 2002a) as well as enough luminous 
galaxies at z ^ 1.5-2 (Cimatti et al. 2002b; van Dokkum et al. 
2003). In addition, semi-analytical models fail completely to 
reproduce the small age difference between early-type galax- 
ies in clusters and the field out to z = 0.6 (van Dokkum et al. 
2001). Our results may suggest an alternative scenario for the 
formation of early-type galaxies, in which they would result 
from the rapid coalescence (multiple merging) of the red galax- 
ies at 2 < z < 4, assuming these red galaxies are indeed grouped 
within single DM halos with large occupation numbers. 

We are planning extended follow-up spectroscopy of this newly 
discovered population in order to directly measure its real space 
clustering and to fully investigate its nature. Further impor- 
tant steps will be the evaluation of the AGN fraction among 
z > 2 red galaxies and their morphology. Work on these as- 
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pects is currently ongoing for the Chandra Deep Field South 
(CDFS/GOODS), using deep Chandra, VLT/ISAAC and ACS 
imaging data. Future SIRTF observations, planned both for 
HDF-S and for CDFS/GOODS, are expected to provide con- 
straints on the mass of z > 2 red galaxies. 
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Table 1 

Summary of clustering measurements in the HDF-S FIRES survey. 
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a In units of 10 -4 , amplitudes at 1 degree. 
b In units of comoving h~ l Mpc. 

c Standard deviation of the photometric redshift distribution. 



